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Abstract Body 



Background / Context: 

With the expansion of regional and national economies into a global marketplace, education has 
even greater importance as a primary factor in allowing young adults to enter the workforce and 
advance economically, as well as to share in the social, health, and other benefits associated with 
education and productive careers. Dropping out of school before completing the normal course 
of secondary education greatly undermines these opportunities and is associated with adverse 
personal and social consequences. Dropout rates in the United States vary by calculation method, 
state, ethnic background, and socioeconomic status (Cataldi, Laird, & KewelRamani, 2009). 
Across all states, the percentage of freshman who did not graduate from high school in four years 
ranges from 13.1% to 44.2% and averages 26.8%. The status dropout rate, which estimates the 
percentage of individuals in a certain age range who are not in high school and have not earned a 
diploma or credential, is slightly lower. In October 2007, the proportion of noninstitutionalized 
18-24 year olds not in school without a diploma or certificate was 8.7%. Males are more likely to 
be dropouts than females (9.8% vs. 7.7%). Status dropout rates are much higher for racial/ethnic 
minorities (21.4% for Hispanics and 8.4% for Blacks vs. 5.3% for Whites). Event dropout rates 
illustrate single year dropout rates for high school students and show that students from low- 
income households drop out of high school more frequently than those from more advantaged 
backgrounds (8.8% for low-income vs. 3.5% for middle income and 0.9% for high income 
students). The National Dropout Prevention Center/Network reports that school dropouts in the 
United States earn an average of $9,245 a year less than those who complete high school, have 
unemployment rates almost 13 percentage points higher than high school graduates, are 
disproportionately represented in prison populations, are more likely to become teen parents, and 
more frequently live in poverty (2009). The consequences of school dropout are even worse for 
minority youth, further exacerbating the economic and structural disadvantage they often 
experience. 

School dropout has implications not only for the lives and opportunities of those who experience 
it, but also has enormous economic and social implications for society at large. For instance, the 
National Dropout Prevention Center/Network (2009) reports that each annual cohort of dropouts 
costs the United States over $200 billion during their lifetime due to lost earnings and unrealized 
tax revenue; and even a 1% increase in high school graduation rates could save over $1 billion in 
incarceration costs. The Organisation for Economic Co-operation and Development (2009) has 
similarly documented the tremendous social and economic gains associated with secondary 
school completion in OECD member countries. 

A relatively large number of intervention and prevention programs in the research literature give 
some attention to reducing dropout rates as a possible outcome. The National Dropout 
Prevention Center/Network, for instance, lists 192 “model programs.” Relatively few of those 
programs, however, bill themselves as dropout programs; many focus on academic performance, 
risk factors for dropout such as absences or truancy, or indirect outcomes like student 
engagement, but may also include dropout reduction as a program objective. The corresponding 
research domain includes evaluations of virtually any program provided to students for which 
dropout rates are measured as an outcome variable, regardless of whether they are billed as 
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dropout programs. To represent the full scope of relevant research on this topic, all such 
programs should be considered in a review of dropout programs. 

There have been a handful of systematic reviews on the effects of prevention and intervention 
programs on school dropout and completion outcomes. However, the restrictive inclusion criteria 
and methodological weaknesses of these reviews preclude any confident conclusions about the 
effectiveness of the broad range of programs with dropout outcomes, or the potential variation of 
effectiveness for different program types or subject populations. For instance, the U.S. 
Department of Education’s What Works Clearinghouse report on dropout prevention found only 
1 5 qualifying studies that reported outcomes on direct measures of staying in school or 
completing school ( http://ies.ed.gov/ nccc/wwc/rcports/dropout/topic/#top ). This report, 
however, restricted discussion to interventions in the United States and did not include a meta- 
analysis of program effectiveness or examine potential moderators of program effectiveness. 
Another review on best practices in dropout prevention summarized the results of 58 studies of 
dropout programs (ICF, International, 2008). That report presented effect sizes primarily for 
individual program types and did not examine potential moderators or examine the influence of 
study method on effect size. The report also presented a narrative review of important variables 
associated with implementation quality, but implementation quality was not analyzed in a meta- 
analysis framework. 

Two other systematic reviews have focused on the effectiveness of prevention and intervention 
programs to reduce school dropout or increase school completion (Klima, Miller, & Nunlist, 
2009; Lehr et al., 2003). In their review, Lehr et al. (2003) identified 17 experimental or quasi- 
experimental studies with enrollment status outcomes. This review was completed seven years 
ago, and thus does not include the most recent studies. The authors did not perform a meta- 
analysis because they felt that the dependent variables differed too greatly across studies to 
create meaningful aggregates. This circumstance prevented the authors from examining the 
differential effectiveness of programs with different treatment or participant characteristics, 
something we plan to do in the proposed systematic review. In a more recent review, Klima et al. 
(2009) identified 22 experimental or quasi-experimental studies with dropout, achievement, and 
truancy outcomes. However, this review excluded programs for general “at-risk” populations of 
students (e.g., minority or low socioeconomic status samples), as well as programs with general 
character-building, social-emotional learning, or delinquency/behavioral improvement 
components. These exclusion criteria therefore limited the conclusions that could be drawn about 
the broader range of programs that aim to influence school dropout and completion outcomes. 
Further, this review only presented mean effect sizes for different types of interventions, and did 
not examine the potential variation of effects for different subject populations. 

The findings of the Klima et al., (2009) and Lehr et al., (2003) reviews have some similarities. 
Both teams highlight the dearth of high-quality research on dropout programs, and mention 
especially the lack of key outcomes such as enrollment (or presence) at school and dropout. Both 
reviews demonstrate that some of the included programs had positive effects on the students 
involved. Lehr and her colleagues do not identify specific programs that were particularly 
effective or ineffective, but focus rather on implementation integrity as a key variable and 
emphasize the importance of strong methodologies for future research on dropout programs. 
Klima and colleagues conclude that the programs they reviewed had overall positive effects on 
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dropout, achievement, and attendance/enrollment. They highlight alternative educational 
programs, such as schools-within-schools, as particularly effective. The Klima review also 
suggests that alternative school programs, that is, programs in separate school facilities, were 
ineffective. Overall, these two reviews identify several important potential moderators that will 
be included in the coding scheme for the proposed review. These include implementation 
quality, treatment modality, and whether programs are housed in typical school facilities or in 
alternative school locations. 

Purpose / Objective / Research Question / Focus of Study: 

The objective of the systematic review is to summarize the available evidence on the effects of 
prevention and intervention programs aimed at primary and secondary students for increasing 
school completion or reducing school dropout. Program effects on the closely related outcomes 
of school attendance (absences, truancy) will also be examined. Moreover, when accompanying 
dropout or attendance outcomes, effects on student engagement, academic performance, and 
school conduct are also included. 

The primary focus of the analyses presented will be the comparative effectiveness of different 
programs and program approaches in an effort to identify those that have the largest and most 
reliable effects on the respective school participation outcomes, especially with regard to 
differences associated with treatment modality, implementation quality, and program location or 
setting. In addition, evidence of differential effects for students with different characteristics will 
be explored, e.g., in relation to age or grade, gender, race/ethnicity, and risk factors. 

Setting: 

Studies eligible for the systematic review involved both school- and community-based programs. 
The initial search criteria allowed for studies from all over the world, published in any language, 
to be included. In the final sample, the majority of studies are US-based but there are several non 
studies in the sample. 

Population / Participants / Subjects: 

Eligible interventions are directed toward school-aged youth, defined as those expected to attend 
preschool to 12 th grade primary and secondary schools, or the equivalent in countries with a 
different grade structure, corresponding to approximately ages 4-18. The age or school 
participation of the sample must be presented in sufficient detail to allow reasonable inference 
that it meets this requirement. Recent dropouts who are between the ages of 18-21 may also be 
included if the program under study is explicitly oriented toward secondary school completion or 
the equivalent. 

General population samples of school-age children are included. Samples from populations 
broadly at risk because of economic disadvantage, individual risk variables, and closely related 
factors are also included (e.g., inner city schools, students from low SES families, teen parents, 
students with poor attendance records, students who have low test scores or who are over-age for 
their grade). 

Intervention / Program / Practice: 

The interventions included in the systematic review must be school-based, school-affiliated, or 
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community-based psychological, educational, or behavioral prevention or intervention programs, 
broadly defined, that involve actions performed with the expectation that they will have 
beneficial effects on student recipients. School-based programs are those that are administered 
under the auspices of school authorities and delivered during school hours. School affiliated 
programs are those that are delivered with the collaboration of school authorities, possibly by 
other agents, e.g., community service providers, and which may take place before or after school 
hours and/or off the school grounds. Community-based programs that are explicitly presented as 
dropout prevention or intervention programs are also included. Table 1 presents a brief listing of 
the most common interventions in the systematic review (insert Table 1 about here). 

Research Design: 

The research reported here is a systematic review and meta-analysis of school dropout 
interventions and follows standard meta-analysis methodology (Cooper, Hedges, & Valentine, 
2009; Hedges, Higgins, Rothstein, & Borenstein, 2009; Higgins & Green, 2008; Lipsey & 
Wilson, 2001). The methods are described briefly below. 

Data Collection and Analysis: 

We developed a detailed set of eligibility criteria to specify eligible interventions, subject 
populations, research designs and outcomes. A comprehensive literature search was performed 
and potentially eligible studies were screened for eligibility. Studies were coded into a 
computerized coding manual on variables related to study methods, the nature of the intervention 
and its implementation, the characteristics of the subject samples, the outcome variables and 
statistical findings, and contextual features such as setting, year of publication, and the like. 

Effect size metrics. We used odds ratios as the effect size metric for dropout and other binary 
outcomes, measured as the odds of school completion/success divided by the odds of school 
dropout/failure. For outcomes measured on a continuous scale we calculated standardized mean 
difference effect sizes, measured as differences in treatment and comparison group means (e.g., 
group differences in average attendance rates). Standardized mean difference effect sizes will be 
adjusted with a small-sample correction factor that provides unbiased estimates of the effect size 
in small samples (Hedges, 1981). All effect sizes are coded such that larger effect sizes represent 
positive outcomes (e.g., less school dropout, higher attendance, less truancy). 

Findings / Results: 

Thus far, we have coded the study characteristics and results for 130 independent samples of 
students. The general characteristics of the programs in the coded studies are as follows: 

• Community-based programs make up 15% of the sample; the remaining 85% are school- 
based or school-affiliated. 

• Program duration averaged 77 weeks. 

• The frequency of treatment varied, but over half of the programs involved daily contact. 

• Implementation quality also varied: 29% experienced clear problems, 15% alluded to 
possible problems, and 56% experienced no problems or mentioned no problems. 

• 75% were high school aged samples; 18% were middle school samples; 10 studies (7%) 
were younger students. 
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• Mixed ethnicity groups of students were common. 65% of the student samples were 
mostly minority students. 

• With the exception of specialized programs for teenage mothers, most programs were 
delivered to mixed gender groups. 

• Nearly all students were at risk for dropping out and were from low socioeconomic status 
families. 

The weighted mean random effects odds ratio for the effects of treatment on high school dropout 
was 1.63 (95% Cl 1.41-1.89). After treatment, the odds of completing school are 1.63 times 
greater for students in prevention programs than for students who received no special 
programming. The average control group dropout rate is 44%. An odds ratio of 1 .63 means that 
dropout programs reduced this rate to about 32%. The distribution of dropout odds ratios 
evidenced significant heterogeneity (Q=238.6*; I 2 = 45.9%). That is, there are between-study 
differences in effect sizes that are greater than would be expected from the subject-level 
sampling error. To identify study characteristics associated with larger or smaller effects, a 
random effects meta-regression analysis was perfonned. The results are shown in Table 2 (please 
insert Table 2 here). 

Conclusions: 

The results of the regression analyses on the studies coded thus far find that unpublished research 
and higher quality methods produce significantly smaller treatment differences. Ethnic mix and 
grade level were not significantly associated with differential effects for treatment. Key program 
characteristics associated with less dropout among treated students included implementation 
quality, shorter duration programs, and community-based programs. When controlling for other 
influences on outcome (e.g., implementation), all program types produced positive results, 
though the mentoring and “other” programs fell short of significance. Attendance monitoring & 
incentives, child care, community service, and school restructuring programs produced best 
results. 

Results will be discussed in terms of implications for policy and practice. Indeed, the ultimate 
objective of the systematic review is to provide school administrators and policymakers with an 
integrative summary of research evidence that is useful for guiding programmatic efforts to 
reduce school dropout and increase school completion. 
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Appendix B. Tables and Figures 



Table 1. Dropout Programs in the Meta-Analysis 


Program 


Description 


N 


Supplemental academic 
services 


Remedial education, tutoring, homework assistance, 
etc. 


9 


School or class restructuring 


Small learning communities, block schedules, career 
academies, small class size. 


40 


College preparation 


College preparatory curriculum, college-oriented 
academic advising. 


5 


Vocational training 


Coursework in secondary school oriented toward 
work or career interests. 


9 


Attendance monitoring & 
financial incentives 


Monitoring and services to increase attendance for 
mixed gender groups or (for teenage moms) 
monitoring + financial incentives to increase 
attendance. 


17 


Child care services 


Child care services provided for teenage moms. 


2 


CBT or skills training 


Generally oriented toward improving self-esteem or 
attitudes about school, or preventing drug use. 


6 


Case management 


Programs revolved around connecting students & 
families with appropriate services. 


13 


Mentoring, counseling 


Programs provided adult mentors or trained 
counselors for students. Though mentors focused 
more on career/work, both mentors and counselors 
dealt with students’ personal issues. 


5 


Community service 


Series of affiliated programs that involve planning 
and carrying out a community service project coupled 
with a weekly life skills curriculum. 


16 
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Table 2. Meta-Regression Results 




B 


P 


P 


Method Variables 


Unpublished 


-.83 


-.47 


.00 


Routine practice 


.01 


.01 


.94 


Method quality 


-.17 


-.19 


.06 


Subject Characteristics 


Ethnic mix 


.17 


.17 


.13 


Grade level 


-.00 


-.01 


.95 


Treatment Characteristics 


Implementation quality 


.26 


.28 


.03 


Tx duration 


-.00 


-.23 


.03 


Community-based 


.91 


.41 


.00 


Program Types 


Supplemental academic (9) 


.90 


.27 


.04 


School restructuring (40) 


1.41 


.81 


.00 


Attendance monitoring mixed (6) 


2.01 


.37 


.00 


Attendance monitoring teenage moms (11) 


1.65 


.58 


.00 


CBT or skills training (6) 


1.35 


.33 


.01 


Vocational (9) 


.77 


.27 


.07 


Case management (13) 


1.03 


.37 


.03 


College prep. (5) 


1.34 


.39 


.00 


Child care (2) 


1.86 


.25 


.00 


Mentoring, counseling (5) 


.54 


.11 


.28 


Community service (16) 


1.74 


.45 


.00 
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Adjusted Mean Odds Ratios by Program Type 



Program Tvoe 


k 


Other 


3 


Mentoring, counseling 


5 


Vocational training 


9 


Supplemental academic 
training 


9 


Case management 


13 


CBT & other skillstraining 


6 


College preparation 


5 


School or class restructuring 


40 


Attendance monitoring and 
$$ teen moms 


11 


Community service 


16 


Attendance monitoring and 
services mixed gender 


6 


Child care 


2 



<> 



0.5 1 2 4 8 16 32 

Odds Ratio 



Figure 1. 

Forest Plot of Adjusted Effect Sizes by Treatment Type 
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